Papers with multi-layer attention mechanisms
Exploiting Deep Representations for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation models typically implement encoder and decoder as multiple layers, but only the top layers are leveraged in the subsequent process, which misses the opportunity to exploit useful information embedded in other layers. |
| Approach: | They propose to expose all of these signals with layer aggregation and multi-layer attention mechanisms and introduce an auxiliary regularization term to encourage different layers to capture diverse information. |
| Outcome: | The proposed approach exposes all of these signals with layer aggregation and multi-layer attention mechanisms on widely-used translation datasets. |